Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extending AI Actions #2537

Open
wants to merge 14 commits into
base: IBX-8689
Choose a base branch
from
Open

Extending AI Actions #2537

wants to merge 14 commits into from

Conversation

mnocon
Copy link
Contributor

@mnocon mnocon commented Nov 11, 2024

Question Answer
JIRA Ticket JIRA: https://issues.ibexa.co/browse/IBX-9077
Versions 4.6 and 5.0
Edition Headless, Experience, Commerce

This PR builds upon the work done in #2473 - and adds the "how to extend" part of the doc, together with references.

I've included the generated PHP API Reference (this is stolen from #2447) - to avoid polluting this PR with too many files the generated reference is extracted to a separate PR (and all the preview links link there).

General preview: preview

New pages or sections:

Testing the use cases locally:

Setup

Run in a project:

git clone https://github.com/ibexa/documentation-developer -b IBX-8689-extending
cp -R documentation-developer/code_samples/ai_actions/ .
composer run post-install-cmd

Use case 1

  1. Configure OpenAI
  2. Publish an image without an alt text
  3. Run php bin/console app:add-alt-text

The image should have alt text generated automatically

Goal: teach readers how to use the AI PHP API

Use case 2

  1. Run llamafile on port 8080
  2. Create an Action Configuration using the LLaVATextToText Handler
  3. Use it in online editor to improve writing

Goal: teach readers how to create a custom Handler (including Action Handler Form options)

Use case 3

  1. Download sample audio files (file1, file2)
  2. Install Whisper and make it available as an executable called whisper
  3. Add a eztext field with transcript identifier to the File Content Type
  4. Create an Action Configuration using the Whisper Handler for the Transcribe Audio Action Type
  5. Start creating a new file and use the AI to generate a transcription

Goal: teach readers how to create a custom Action Type, including:

  • Action Type
  • Form for Action Type options
  • custom Handler
  • integration with the REST API
  • integration with the back office

TODO:

  • Add the optional pill to the namespaces (not only to classes)
  • Follow up: fix the PHP reference errors

Checklist

  • Text renders correctly
  • Text has been checked with vale
  • Description metadata is up to date
  • Code samples are working
  • PHP code samples have been fixed with PHP CS fixer
  • Added link to this PR in relevant JIRA ticket or code PR

@mnocon mnocon changed the base branch from master to IBX-8689 November 11, 2024 23:10
Copy link

code_samples/ change report

Before (on target branch)After (in current PR)

code_samples/ai_actions/assets/js/addAudioModule.js


code_samples/ai_actions/assets/js/addAudioModule.js

docs/ai_actions/extend_ai_actions.md@323:``` js
docs/ai_actions/extend_ai_actions.md@324:[[= include_file('code_samples/ai_actions/assets/js/addAudioModule.js') =]]
docs/ai_actions/extend_ai_actions.md@325:```

001⫶import { addModule } from '../../vendor/ibexa/connector-ai/src/bundle/Resources/public/js/core/create.ai.module';
002⫶import { default as TranscribeAudio } from './transcribe.audio'
003⫶
004⫶addModule(TranscribeAudio );


code_samples/ai_actions/assets/js/transcribe.audio.js


code_samples/ai_actions/assets/js/transcribe.audio.js

docs/ai_actions/extend_ai_actions.md@315:``` js
docs/ai_actions/extend_ai_actions.md@316:[[= include_file('code_samples/ai_actions/assets/js/transcribe.audio.js') =]]
docs/ai_actions/extend_ai_actions.md@317:```

001⫶import BaseAIComponent from '../../vendor/ibexa/connector-ai/src/bundle/Resources/public/js/core/base.ai.component';
002⫶
003⫶export default class TranscribeAudio extends BaseAIComponent {
004⫶ constructor(mainElement, config) {
005⫶ super(mainElement, config);
006⫶
007⫶ this.requestHeaders = {
008⫶ Accept: 'application/vnd.ibexa.api.ai.AudioText+json',
009⫶ 'Content-Type': 'application/vnd.ibexa.api.ai.TranscribeAudio+json',
010⫶ };
011⫶ }
012⫶
013⫶ getAudioInBase64() {
014⫶ var request = new XMLHttpRequest();
015⫶ request.open('GET', this.inputElement.href, false);
016⫶ request.overrideMimeType('text\/plain; charset=x-user-defined');
017⫶ request.send();
018⫶
019⫶ if (request.status === 200) {
020⫶ return this.convertToBase64(request.responseText);
021⫶ }
022⫶ }
023⫶
024⫶ getRequestBody() {
025⫶ const body = {
026⫶ TranscribeAudio: {
027⫶ Audio: {
028⫶ base64: this.getAudioInBase64(),
029⫶ },
030⫶ RuntimeContext: {},
031⫶ },
032⫶ };
033⫶
034⫶ if (this.languageCode) {
035⫶ body.TranscribeAudio.RuntimeContext.languageCode = this.languageCode;
036⫶ }
037⫶
038⫶ return JSON.stringify(body);
039⫶ }
040⫶
041⫶ afterFetchData(response) {
042⫶ super.afterFetchData();
043⫶
044⫶ if (response) {
045⫶ this.outputElement.value = response.AudioText.Text.text[0];
046⫶ }
047⫶ }
048⫶
049⫶ toggle(forceEnabled) {
050⫶ super.toggle(forceEnabled);
051⫶
052⫶ this.outputElement.disabled = !forceEnabled || !this.outputElement.disabled;
053⫶ }
054⫶
055⫶ convertToBase64(data) {
056⫶ var binary = ""
057⫶
058⫶ for(var i=0;i
059⫶ binary += String.fromCharCode(data.charCodeAt(i) & 0xff);
060⫶ }
061⫶
062⫶ return btoa(binary);
063⫶ }
064⫶}


code_samples/ai_actions/config/packages/ibexa_admin_ui.yaml


code_samples/ai_actions/config/packages/ibexa_admin_ui.yaml

docs/ai_actions/extend_ai_actions.md@294:``` yaml
docs/ai_actions/extend_ai_actions.md@295:[[= include_file('code_samples/ai_actions/config/packages/ibexa_admin_ui.yaml', 4, 5) =]][[= include_file('code_samples/ai_actions/config/packages/ibexa_admin_ui.yaml', 13, 15) =]][[= include_file('code_samples/ai_actions/config/packages/ibexa_admin_ui.yaml', 84) =]]
docs/ai_actions/extend_ai_actions.md@296:```

001⫶ibexa:
002⫶ system:
003⫶ admin_group:


code_samples/ai_actions/config/services.yaml


code_samples/ai_actions/config/services.yaml

docs/ai_actions/extend_ai_actions.md@45:``` yaml
docs/ai_actions/extend_ai_actions.md@46:[[= include_file('code_samples/ai_actions/config/services.yaml', 25, 28) =]]
docs/ai_actions/extend_ai_actions.md@47:```

001⫶ App\Command\AddMissingAltTextCommand:
002⫶ arguments:
003⫶ $projectDir: '%kernel.project_dir%'

docs/ai_actions/extend_ai_actions.md@112:``` yaml
docs/ai_actions/extend_ai_actions.md@113:[[= include_file('code_samples/ai_actions/config/services.yaml', 33, 37) =]]
docs/ai_actions/extend_ai_actions.md@114:```

001⫶ app.connector_ai.action_configuration.handler.llava_text_to_text.form_mapper.options:
002⫶ class: Ibexa\Bundle\ConnectorAi\Form\FormMapper\ActionConfiguration\ActionHandlerOptionsFormMapper
003⫶ arguments:

docs/ai_actions/extend_ai_actions.md@132:``` yaml
docs/ai_actions/extend_ai_actions.md@133:[[= include_file('code_samples/ai_actions/config/services.yaml', 38, 45) =]]
docs/ai_actions/extend_ai_actions.md@134:```

001⫶ tags:
002⫶ - name: ibexa.connector_ai.action_configuration.form_mapper.options
003⫶ type: !php/const \App\AI\Handler\LLaVaTextToTextActionHandler::IDENTIFIER
004⫶
005⫶ App\AI\ActionType\TranscribeAudioActionType:
006⫶ arguments:
007⫶ $actionHandlers: !tagged_iterator

docs/ai_actions/extend_ai_actions.md@158:``` yaml
docs/ai_actions/extend_ai_actions.md@159:[[= include_file('code_samples/ai_actions/config/services.yaml', 42, 50) =]]
docs/ai_actions/extend_ai_actions.md@160:```

001⫶ App\AI\ActionType\TranscribeAudioActionType:
002⫶ arguments:
003⫶ $actionHandlers: !tagged_iterator
004⫶ tag: app.connector_ai.action.handler.audio_to_text
005⫶ default_index_method: getIdentifier
006⫶ index_by: key
007⫶ tags:
008⫶ - { name: ibexa.ai.action.type, identifier: !php/const \App\AI\ActionType\TranscribeAudioActionType::IDENTIFIER }

docs/ai_actions/extend_ai_actions.md@192:``` yaml
docs/ai_actions/extend_ai_actions.md@193:[[= include_file('code_samples/ai_actions/config/services.yaml', 51, 58) =]]
docs/ai_actions/extend_ai_actions.md@194:```

001⫶ app.connector_ai.action_configuration.handler.transcribe_audio.form_mapper.options:
002⫶ class: Ibexa\Bundle\ConnectorAi\Form\FormMapper\ActionConfiguration\ActionTypeOptionsFormMapper
003⫶ arguments:
004⫶ $formType: 'App\Form\Type\TranscribeAudioOptionsType'
005⫶ tags:
006⫶ - name: ibexa.connector_ai.action_configuration.form_mapper.action_type_options
007⫶ type: !php/const \App\AI\ActionType\TranscribeAudioActionType::IDENTIFIER

docs/ai_actions/extend_ai_actions.md@206:``` yaml
docs/ai_actions/extend_ai_actions.md@207:[[= include_file('code_samples/ai_actions/config/services.yaml', 59, 63) =]]
docs/ai_actions/extend_ai_actions.md@208:```

001⫶ App\AI\Handler\WhisperAudioToTextActionHandler:
002⫶ tags:
003⫶ - { name: ibexa.ai.action.handler, priority: 0 }
004⫶ - { name: app.connector_ai.action.handler.audio_to_text, priority: 0 }

docs/ai_actions/extend_ai_actions.md@221:``` yaml
docs/ai_actions/extend_ai_actions.md@222:[[= include_file('code_samples/ai_actions/config/services.yaml', 65, 69) =]]
docs/ai_actions/extend_ai_actions.md@223:```

001⫶ App\AI\REST\Input\Parser\TranscribeAudio:
002⫶ parent: Ibexa\ConnectorAi\REST\Input\Parser\Action
003⫶ autowire: true
004⫶ tags:

docs/ai_actions/extend_ai_actions.md@246:``` yaml
docs/ai_actions/extend_ai_actions.md@247:[[= include_file('code_samples/ai_actions/config/services.yaml', 70, 73) =]]
docs/ai_actions/extend_ai_actions.md@248:```

001⫶ App\AI\REST\Output\Resolver\AudioTextResolver:
002⫶ tags:

docs/ai_actions/extend_ai_actions.md@257:``` yaml
docs/ai_actions/extend_ai_actions.md@258:[[= include_file('code_samples/ai_actions/config/services.yaml', 74, 78) =]]
docs/ai_actions/extend_ai_actions.md@259:```

001⫶ App\AI\REST\Output\ValueObjectVisitor\AudioText:
002⫶ parent: Ibexa\Contracts\Rest\Output\ValueObjectVisitor
003⫶ tags:


code_samples/ai_actions/src/AI/Action/TranscribeAudioAction.php


code_samples/ai_actions/src/AI/Action/TranscribeAudioAction.php

docs/ai_actions/extend_ai_actions.md@180:``` php
docs/ai_actions/extend_ai_actions.md@181:[[= include_file('code_samples/ai_actions/src/AI/Action/TranscribeAudioAction.php') =]]
docs/ai_actions/extend_ai_actions.md@182:```

001⫶
002⫶
003⫶namespace App\AI\Action;
004⫶
005⫶use App\AI\DataType\Audio;
006⫶use Ibexa\Contracts\ConnectorAi\Action\Action;
007⫶
008⫶final class TranscribeAudioAction extends Action
009⫶{
010⫶ private Audio $audio;
011⫶
012⫶ public function __construct(Audio $audio)
013⫶ {
014⫶ $this->audio = $audio;
015⫶ }
016⫶
017⫶ public function getParameters(): array
018⫶ {
019⫶ return [];
020⫶ }
021⫶
022⫶ public function getInput(): Audio
023⫶ {
024⫶ return $this->audio;
025⫶ }
026⫶
027⫶ public function getActionTypeIdentifier(): string
028⫶ {
029⫶ return 'transcribe_audio';
030⫶ }
031⫶}


code_samples/ai_actions/src/AI/ActionType/TranscribeAudioActionType.php


code_samples/ai_actions/src/AI/ActionType/TranscribeAudioActionType.php

docs/ai_actions/extend_ai_actions.md@154:``` php
docs/ai_actions/extend_ai_actions.md@155:[[= include_file('code_samples/ai_actions/src/AI/ActionType/TranscribeAudioActionType.php') =]]
docs/ai_actions/extend_ai_actions.md@156:```

001⫶
002⫶
003⫶namespace App\AI\ActionType;
004⫶
005⫶use App\AI\Action\TranscribeAudioAction;
006⫶use App\AI\DataType\Audio;
007⫶use Ibexa\Contracts\ConnectorAi\Action\DataType\Text;
008⫶use Ibexa\Contracts\ConnectorAi\ActionInterface;
009⫶use Ibexa\Contracts\ConnectorAi\ActionType\ActionTypeInterface;
010⫶use Ibexa\Contracts\ConnectorAi\DataType;
011⫶use Ibexa\Contracts\Core\Exception\InvalidArgumentException;
012⫶
013⫶final class TranscribeAudioActionType implements ActionTypeInterface
014⫶{
015⫶ public const IDENTIFIER = 'transcribe_audio';
016⫶
017⫶ /** @var iterable<\Ibexa\Contracts\ConnectorAi\Action\ActionHandlerInterface> */
018⫶ private iterable $actionHandlers;
019⫶
020⫶ /** @param iterable<\Ibexa\Contracts\ConnectorAi\Action\ActionHandlerInterface> $actionHandlers*/
021⫶ public function __construct(iterable $actionHandlers)
022⫶ {
023⫶ $this->actionHandlers = $actionHandlers;
024⫶ }
025⫶
026⫶ public function getIdentifier(): string
027⫶ {
028⫶ return self::IDENTIFIER;
029⫶ }
030⫶
031⫶ public function getName(): string
032⫶ {
033⫶ return 'Transcribe audio';
034⫶ }
035⫶
036⫶ public function getInputIdentifier(): string
037⫶ {
038⫶ return Audio::getIdentifier();
039⫶ }
040⫶
041⫶ public function getOutputIdentifier(): string
042⫶ {
043⫶ return Text::getIdentifier();
044⫶ }
045⫶
046⫶ public function getOptions(): array
047⫶ {
048⫶ return [];
049⫶ }
050⫶
051⫶ public function createAction(DataType $input, array $parameters = []): ActionInterface
052⫶ {
053⫶ if (!$input instanceof Audio) {
054⫶ throw new InvalidArgumentException(
055⫶ 'audio',
056⫶ 'expected \App\AI\DataType\Audio type, ' . get_debug_type($input) . ' given.'
057⫶ );
058⫶ }
059⫶
060⫶ return new TranscribeAudioAction($input);
061⫶ }
062⫶
063⫶ public function getActionHandlers(): iterable
064⫶ {
065⫶ return $this->actionHandlers;
066⫶ }
067⫶}


code_samples/ai_actions/src/AI/DataType/Audio.php


code_samples/ai_actions/src/AI/DataType/Audio.php

docs/ai_actions/extend_ai_actions.md@174:``` php
docs/ai_actions/extend_ai_actions.md@175:[[= include_file('code_samples/ai_actions/src/AI/DataType/Audio.php') =]]
docs/ai_actions/extend_ai_actions.md@176:```

001⫶
002⫶
003⫶namespace App\AI\DataType;
004⫶
005⫶use Ibexa\Contracts\ConnectorAi\DataType;
006⫶
007⫶/**
008⫶ * @implements DataType
009⫶ */
010⫶final class Audio implements DataType
011⫶{
012⫶ /** @var non-empty-array */
013⫶ private array $base64;
014⫶
015⫶ /**
016⫶ * @param non-empty-array $base64
017⫶ */
018⫶ public function __construct(array $base64)
019⫶ {
020⫶ $this->base64 = $base64;
021⫶ }
022⫶
023⫶ public function getBase64(): string
024⫶ {
025⫶ return reset($this->base64);
026⫶ }
027⫶
028⫶ public function getList(): array
029⫶ {
030⫶ return $this->base64;
031⫶ }
032⫶
033⫶ public static function getIdentifier(): string
034⫶ {
035⫶ return 'audio';
036⫶ }
037⫶}


code_samples/ai_actions/src/AI/Handler/LLaVaTextToTextActionHandler.php


code_samples/ai_actions/src/AI/Handler/LLaVaTextToTextActionHandler.php

docs/ai_actions/extend_ai_actions.md@108:``` php hl_lines="27-30 32-67 69-72"
docs/ai_actions/extend_ai_actions.md@109:[[= include_file('code_samples/ai_actions/src/AI/Handler/LLaVaTextToTextActionHandler.php') =]]
docs/ai_actions/extend_ai_actions.md@110:```

001⫶
002⫶
003⫶namespace App\AI\Handler;
004⫶
005⫶use Ibexa\Contracts\ConnectorAi\Action\ActionHandlerInterface;
006⫶use Ibexa\Contracts\ConnectorAi\Action\DataType\Text;
007⫶use Ibexa\Contracts\ConnectorAi\Action\TextToText\Action as TextToTextAction;
008⫶use Ibexa\Contracts\ConnectorAi\Action\TextToText\ActionResponse as TextToTextActionResponse;
009⫶use Ibexa\Contracts\ConnectorAi\ActionInterface;
010⫶use Ibexa\Contracts\ConnectorAi\ActionResponseInterface;
011⫶use Symfony\Contracts\HttpClient\HttpClientInterface;
012⫶
013⫶final class LLaVaTextToTextActionHandler implements ActionHandlerInterface
014⫶{
015⫶ private HttpClientInterface $client;
016⫶
017⫶ private string $host;
018⫶
019⫶ public const string IDENTIFIER = 'LLaVATextToText';
020⫶
021⫶ public function __construct(HttpClientInterface $client, string $host = 'http://localhost:8080')
022⫶ {
023⫶ $this->client = $client;
024⫶ $this->host = $host;
025⫶ }
026⫶
027⫸ public function supports(ActionInterface $action): bool
028⫸ {
029⫸ return $action instanceof TextToTextAction;
030⫸ }
031⫶
032⫸ public function handle(ActionInterface $action, array $context = []): ActionResponseInterface
033⫸ {
034⫸ /** @var \Ibexa\Contracts\ConnectorAi\Action\DataType\Text */
035⫸ $input = $action->getInput();
036⫸ $text = $this->sanitizeInput($input->getText());
037⫸
038⫸ $systemMessage = $action->hasActionContext() ? $action->getActionContext()->getActionHandlerOptions()->get('system_prompt', '') : '';
039⫸
040⫸ $response = $this->client->request(
041⫸ 'POST',
042⫸ sprintf('%s/v1/chat/completions', $this->host),
043⫸ [
044⫸ 'headers' => [
045⫸ 'Authorization: Bearer no-key',
046⫸ ],
047⫸ 'json' => [
048⫸ 'model' => 'LLaMA_CPP',
049⫸ 'messages' => [
050⫸ (object)[
051⫸ 'role' => 'system',
052⫸ 'content' => $systemMessage,
053⫸ ],
054⫸ (object)[
055⫸ 'role' => 'user',
056⫸ 'content' => $text,
057⫸ ],
058⫸ ],
059⫸ 'temperature' => 0.7,
060⫸ ],
061⫸ ]
062⫸ );
063⫸
064⫸ $output = strip_tags(json_decode($response->getContent(), true)['choices'][0]['message']['content']);
065⫸
066⫸ return new TextToTextActionResponse(new Text([$output]));
067⫸ }
068⫶
069⫸ public static function getIdentifier(): string
070⫸ {
071⫸ return self::IDENTIFIER;
072⫸ }
073⫶
074⫶ private function sanitizeInput(string $text): string
075⫶ {
076⫶ return str_replace(["\n", "\r"], ' ', $text);
077⫶ }
078⫶}


code_samples/ai_actions/src/AI/Handler/WhisperAudioToTextActionHandler.php


code_samples/ai_actions/src/AI/Handler/WhisperAudioToTextActionHandler.php

docs/ai_actions/extend_ai_actions.md@202:``` php hl_lines="30-33 48"
docs/ai_actions/extend_ai_actions.md@203:[[= include_file('code_samples/ai_actions/src/AI/Handler/WhisperAudioToTextActionHandler.php') =]]
docs/ai_actions/extend_ai_actions.md@204:```

001⫶
002⫶
003⫶namespace App\AI\Handler;
004⫶
005⫶use App\AI\ActionType\TranscribeAudioActionType;
006⫶use Ibexa\Contracts\ConnectorAi\Action\ActionHandlerInterface;
007⫶use Ibexa\Contracts\ConnectorAi\Action\DataType\Text;
008⫶use Ibexa\Contracts\ConnectorAi\Action\TextToText\ActionResponse;
009⫶use Ibexa\Contracts\ConnectorAi\ActionInterface;
010⫶use Ibexa\Contracts\ConnectorAi\ActionResponseInterface;
011⫶use Symfony\Component\Process\Exception\ProcessFailedException;
012⫶use Symfony\Component\Process\Process;
013⫶
014⫶final class WhisperAudioToTextActionHandler implements ActionHandlerInterface
015⫶{
016⫶ public function supports(ActionInterface $action): bool
017⫶ {
018⫶ return $action->getActionTypeIdentifier() === TranscribeAudioActionType::IDENTIFIER;
019⫶ }
020⫶
021⫶ public function handle(ActionInterface $action, array $context = []): ActionResponseInterface
022⫶ {
023⫶ /** @var \App\AI\DataType\Audio $input */
024⫶ $input = $action->getInput();
025⫶
026⫶ $path = $this->saveInputToFile($input->getBase64());
027⫶
028⫶ $arguments = ['whisper'];
029⫶
030⫸ $language = $action->getRuntimeContext()?->get('languageCode');
031⫸ if ($language !== null) {
032⫸ $arguments[] = sprintf('--language=%s', substr($language, 0, 2));
033⫸ }
034⫶
035⫶ $arguments[] = '--output_format=txt';
036⫶ $arguments[] = $path;
037⫶
038⫶ $process = new Process($arguments);
039⫶ $process->run();
040⫶
041⫶ if (!$process->isSuccessful()) {
042⫶ unlink($path);
043⫶ throw new ProcessFailedException($process);
044⫶ }
045⫶
046⫶ $output = $process->getOutput();
047⫶
048⫸ $includeTimestamps = $action->getActionContext()?->getActionTypeOptions()->get('include_timestamps', false) ?? false;
049⫶
050⫶ if (!$includeTimestamps) {
051⫶ $output = $this->removeTimestamps($output);
052⫶ }
053⫶
054⫶ unlink($path);
055⫶
056⫶ return new ActionResponse(new Text([$output]));
057⫶ }
058⫶
059⫶ public static function getIdentifier(): string
060⫶ {
061⫶ return 'whisper_audio_to_text';
062⫶ }
063⫶
064⫶ private function removeTimestamps(string $text): string
065⫶ {
066⫶ $lines = explode(PHP_EOL, $text);
067⫶
068⫶ $processed_lines = array_map(static function (string $line) {
069⫶ return preg_replace('/^\[\d{2}:\d{2}\.\d{3} --> \d{2}:\d{2}\.\d{3}]\s*/', '', $line);
070⫶ }, $lines);
071⫶
072⫶ return implode(PHP_EOL, $processed_lines);
073⫶ }
074⫶
075⫶ private function saveInputToFile(string $audioEncodedInBase64): string
076⫶ {
077⫶ $filename = uniqid('audio');
078⫶ $path = sys_get_temp_dir() . \DIRECTORY_SEPARATOR . $filename;
079⫶ file_put_contents($path, base64_decode($audioEncodedInBase64));
080⫶
081⫶ return $path;
082⫶ }
083⫶}


code_samples/ai_actions/src/AI/REST/Input/Parser/TranscribeAudio.php


code_samples/ai_actions/src/AI/REST/Input/Parser/TranscribeAudio.php

docs/ai_actions/extend_ai_actions.md@217:``` php
docs/ai_actions/extend_ai_actions.md@218:[[= include_file('code_samples/ai_actions/src/AI/REST/Input/Parser/TranscribeAudio.php') =]]
docs/ai_actions/extend_ai_actions.md@219:```

001⫶
002⫶
003⫶namespace App\AI\REST\Input\Parser;
004⫶
005⫶use App\AI\DataType\Audio as AudioDataType;
006⫶use App\AI\REST\Value\TranscribeAudioAction;
007⫶use Ibexa\ConnectorAi\REST\Input\Parser\Action;
008⫶use Ibexa\Contracts\Rest\Input\ParsingDispatcher;
009⫶
010⫶final class TranscribeAudio extends Action
011⫶{
012⫶ public const AUDIO_KEY = 'Audio';
013⫶ public const BASE64_KEY = 'base64';
014⫶
015⫶ public function parse(array $data, ParsingDispatcher $parsingDispatcher): TranscribeAudioAction
016⫶ {
017⫶ $this->assertInputIsValid($data);
018⫶ $runtimeContext = $this->getRuntimeContext($data);
019⫶
020⫶ return new TranscribeAudioAction(
021⫶ new AudioDataType([$data[self::AUDIO_KEY][self::BASE64_KEY]]),
022⫶ $runtimeContext
023⫶ );
024⫶ }
025⫶
026⫶ /** @param array $data */
027⫶ private function assertInputIsValid(array $data): void
028⫶ {
029⫶ if (!array_key_exists(self::AUDIO_KEY, $data)) {
030⫶ throw new \InvalidArgumentException('Missing audio key');
031⫶ }
032⫶
033⫶ if (!array_key_exists(self::BASE64_KEY, $data[self::AUDIO_KEY])) {
034⫶ throw new \InvalidArgumentException('Missing base64 key');
035⫶ }
036⫶ }
037⫶}


code_samples/ai_actions/src/AI/REST/Output/Resolver/AudioTextResolver.php


code_samples/ai_actions/src/AI/REST/Output/Resolver/AudioTextResolver.php

docs/ai_actions/extend_ai_actions.md@242:``` php
docs/ai_actions/extend_ai_actions.md@243:[[= include_file('code_samples/ai_actions/src/AI/REST/Output/Resolver/AudioTextResolver.php') =]]
docs/ai_actions/extend_ai_actions.md@244:```

001⫶
002⫶
003⫶namespace App\AI\REST\Output\Resolver;
004⫶
005⫶use App\AI\REST\Value\AudioText;
006⫶use Ibexa\ConnectorAi\REST\Output\ResolverInterface;
007⫶use Ibexa\Contracts\ConnectorAi\ActionResponseInterface;
008⫶
009⫶final class AudioTextResolver implements ResolverInterface
010⫶{
011⫶ public function getRestValue(
012⫶ ActionResponseInterface $actionResponse
013⫶ ): AudioText {
014⫶ return new AudioText(
015⫶ $actionResponse->getOutput()
016⫶ );
017⫶ }
018⫶}


code_samples/ai_actions/src/AI/REST/Output/ValueObjectVisitor/AudioText.php


code_samples/ai_actions/src/AI/REST/Output/ValueObjectVisitor/AudioText.php

docs/ai_actions/extend_ai_actions.md@253:``` php
docs/ai_actions/extend_ai_actions.md@254:[[= include_file('code_samples/ai_actions/src/AI/REST/Output/ValueObjectVisitor/AudioText.php') =]]
docs/ai_actions/extend_ai_actions.md@255:```

001⫶
002⫶
003⫶namespace App\AI\REST\Output\ValueObjectVisitor;
004⫶
005⫶use Ibexa\Contracts\Rest\Output\Generator;
006⫶use Ibexa\Contracts\Rest\Output\ValueObjectVisitor;
007⫶use Ibexa\Contracts\Rest\Output\Visitor;
008⫶
009⫶final class AudioText extends ValueObjectVisitor
010⫶{
011⫶ private const OBJECT_IDENTIFIER = 'AudioText';
012⫶
013⫶ public function visit(Visitor $visitor, Generator $generator, $data): void
014⫶ {
015⫶ $mediaType = 'ai.' . self::OBJECT_IDENTIFIER;
016⫶ $text = $data->getOutput();
017⫶
018⫶ $generator->startObjectElement(self::OBJECT_IDENTIFIER, $mediaType);
019⫶ $visitor->setHeader('Content-Type', $generator->getMediaType($mediaType));
020⫶
021⫶ $visitor->visitValueObject($text);
022⫶
023⫶ $generator->endObjectElement(self::OBJECT_IDENTIFIER);
024⫶ }
025⫶}


code_samples/ai_actions/src/AI/REST/Value/AudioText.php


code_samples/ai_actions/src/AI/REST/Value/AudioText.php

docs/ai_actions/extend_ai_actions.md@236:``` php
docs/ai_actions/extend_ai_actions.md@237:[[= include_file('code_samples/ai_actions/src/AI/REST/Value/AudioText.php') =]]
docs/ai_actions/extend_ai_actions.md@238:```

001⫶
002⫶
003⫶namespace App\AI\REST\Value;
004⫶
005⫶use Ibexa\ConnectorAi\REST\Value\RestActionResponse;
006⫶
007⫶final class AudioText extends RestActionResponse
008⫶{
009⫶}


code_samples/ai_actions/src/AI/REST/Value/TranscribeAudioAction.php


code_samples/ai_actions/src/AI/REST/Value/TranscribeAudioAction.php

docs/ai_actions/extend_ai_actions.md@227:``` php
docs/ai_actions/extend_ai_actions.md@228:[[= include_file('code_samples/ai_actions/src/AI/REST/Value/TranscribeAudioAction.php') =]]
docs/ai_actions/extend_ai_actions.md@229:```

001⫶
002⫶
003⫶namespace App\AI\REST\Value;
004⫶
005⫶use Ibexa\ConnectorAi\REST\Value\RestAction;
006⫶
007⫶final class TranscribeAudioAction extends RestAction
008⫶{
009⫶}


code_samples/ai_actions/src/Command/ActionConfigurationCreateCommand.php


code_samples/ai_actions/src/Command/ActionConfigurationCreateCommand.php

docs/ai_actions/extend_ai_actions.md@71:``` php hl_lines="3 17"
docs/ai_actions/extend_ai_actions.md@72:[[= include_file('code_samples/ai_actions/src/Command/ActionConfigurationCreateCommand.php', 57, 74) =]]
docs/ai_actions/extend_ai_actions.md@73:```

001⫶ $refineTextActionType = $this->actionTypeRegistry->getActionType('refine_text');
002⫶
003⫸ $actionConfigurationCreateStruct = new ActionConfigurationCreateStruct('rewrite_casual');
004⫶
005⫶ $actionConfigurationCreateStruct->setType($refineTextActionType);
006⫶ $actionConfigurationCreateStruct->setName('eng-GB', 'Rewrite in casual tone');
007⫶ $actionConfigurationCreateStruct->setDescription('eng-GB', 'Rewrites the text using a casual tone');
008⫶ $actionConfigurationCreateStruct->setActionHandler('openai-text-to-text');
009⫶ $actionConfigurationCreateStruct->setActionHandlerOptions(new ArrayMap([
010⫶ 'max_tokens' => 4000,
011⫶ 'temperature' => 1,
012⫶ 'prompt' => 'Rewrite this content to improve readability. Preserve meaning and crucial information but use casual language accessible to a broader audience.',
013⫶ 'model' => 'gpt-4-turbo',
014⫶ ]));
015⫶ $actionConfigurationCreateStruct->setEnabled(true);
016⫶
017⫸ $this->actionConfigurationService->createActionConfiguration($actionConfigurationCreateStruct);

docs/ai_actions/extend_ai_actions.md@81:``` php hl_lines="7-8"
docs/ai_actions/extend_ai_actions.md@82:[[= include_file('code_samples/ai_actions/src/Command/ActionConfigurationCreateCommand.php', 75, 83) =]]
docs/ai_actions/extend_ai_actions.md@83:```

001⫶ $action = new RefineTextAction(new Text([
002⫶ <<
003⫶ Proteins differ from one another primarily in their sequence of amino acids, which is dictated by the nucleotide sequence of their genes,
004⫶ and which usually results in protein folding into a specific 3D structure that determines its activity.
005⫶TEXT
006⫶ ]));
007⫸ $actionConfiguration = $this->actionConfigurationService->getActionConfiguration('rewrite_casual');
008⫸ $actionResponse = $this->actionService->execute($action, $actionConfiguration)->getOutput();


code_samples/ai_actions/src/Command/AddMissingAltTextCommand.php


code_samples/ai_actions/src/Command/AddMissingAltTextCommand.php

docs/ai_actions/extend_ai_actions.md@15:``` php
docs/ai_actions/extend_ai_actions.md@16:[[= include_file('code_samples/ai_actions/src/Command/AddMissingAltTextCommand.php', 99, 118) =]]
docs/ai_actions/extend_ai_actions.md@17:```

001⫶ $action = new GenerateAltTextAction(new Image([$imageEncodedInBase64]));
002⫶
003⫶ $action->setRuntimeContext(new RuntimeContext(['languageCode' => $languageCode]));
004⫶ $action->setActionContext(
005⫶ new ActionContext(
006⫶ new Options(['default_locale_fallback' => 'en']), // System context
007⫶ new Options(['max_lenght' => 100]), // Action Type options
008⫶ new Options( // Action Handler options
009⫶ [
010⫶ 'prompt' => 'Generate the alt text for this image in less than 100 characters.',
011⫶ 'temperature' => 0.7,
012⫶ 'max_tokens' => 4096,
013⫶ 'model' => 'gpt-4o-mini',
014⫶ ]
015⫶ )
016⫶ )
017⫶ );
018⫶
019⫶ $output = $this->actionService->execute($action)->getOutput();

docs/ai_actions/extend_ai_actions.md@41:``` php hl_lines="85 98-123"
docs/ai_actions/extend_ai_actions.md@42:[[= include_file('code_samples/ai_actions/src/Command/AddMissingAltTextCommand.php') =]]
docs/ai_actions/extend_ai_actions.md@43:```

001⫶
002⫶
003⫶namespace App\Command;
004⫶
005⫶use Ibexa\AdminUi\Event\Options;
006⫶use Ibexa\Contracts\ConnectorAi\Action\ActionContext;
007⫶use Ibexa\Contracts\ConnectorAi\Action\DataType\Image;
008⫶use Ibexa\Contracts\ConnectorAi\Action\DataType\Text;
009⫶use Ibexa\Contracts\ConnectorAi\Action\GenerateAltTextAction;
010⫶use Ibexa\Contracts\ConnectorAi\Action\RuntimeContext;
011⫶use Ibexa\Contracts\ConnectorAi\ActionServiceInterface;
012⫶use Ibexa\Contracts\Core\Repository\ContentService;
013⫶use Ibexa\Contracts\Core\Repository\FieldTypeService;
014⫶use Ibexa\Contracts\Core\Repository\PermissionResolver;
015⫶use Ibexa\Contracts\Core\Repository\UserService;
016⫶use Ibexa\Contracts\Core\Repository\Values\Content\ContentList;
017⫶use Ibexa\Contracts\Core\Repository\Values\Content\Query\Criterion\ContentTypeIdentifier;
018⫶use Ibexa\Contracts\Core\Repository\Values\Content\Query\Criterion\DateMetadata;
019⫶use Ibexa\Contracts\Core\Repository\Values\Content\Query\Criterion\Operator;
020⫶use Ibexa\Contracts\Core\Repository\Values\Filter\Filter;
021⫶use Ibexa\Core\FieldType\Image\Value;
022⫶use Symfony\Component\Console\Command\Command;
023⫶use Symfony\Component\Console\Input\InputArgument;
024⫶use Symfony\Component\Console\Input\InputInterface;
025⫶use Symfony\Component\Console\Output\OutputInterface;
026⫶
027⫶final class AddMissingAltTextCommand extends Command
028⫶{
029⫶ protected static $defaultName = 'app:add-alt-text';
030⫶
031⫶ private const IMAGE_FIELD_IDENTIFIER = 'image';
032⫶
033⫶ private ContentService $contentService;
034⫶
035⫶ private PermissionResolver $permissionResolver;
036⫶
037⫶ private UserService $userService;
038⫶
039⫶ private FieldTypeService $fieldTypeService;
040⫶
041⫶ private ActionServiceInterface $actionService;
042⫶
043⫶ private string $projectDir;
044⫶
045⫶ public function __construct(
046⫶ ContentService $contentService,
047⫶ PermissionResolver $permissionResolver,
048⫶ UserService $userService,
049⫶ FieldTypeService $fieldTypeService,
050⫶ ActionServiceInterface $actionService,
051⫶ string $projectDir
052⫶ ) {
053⫶ parent::__construct();
054⫶ $this->contentService = $contentService;
055⫶ $this->permissionResolver = $permissionResolver;
056⫶ $this->userService = $userService;
057⫶ $this->fieldTypeService = $fieldTypeService;
058⫶ $this->actionService = $actionService;
059⫶ $this->projectDir = $projectDir;
060⫶ }
061⫶
062⫶ protected function configure(): void
063⫶ {
064⫶ $this->addArgument('user', InputArgument::OPTIONAL, 'Login of the user executing the actions', 'admin');
065⫶ }
066⫶
067⫶ protected function execute(InputInterface $input, OutputInterface $output): int
068⫶ {
069⫶ $this->setUser($input->getArgument('user'));
070⫶
071⫶ $modifiedImages = $this->getModifiedImages();
072⫶ $output->writeln(sprintf('Found %d modified image in the last 24h', $modifiedImages->getTotalCount()));
073⫶
074⫶ /** @var \Ibexa\Core\Repository\Values\Content\Content $content */
075⫶ foreach ($modifiedImages as $content) {
076⫶ /** @var ?Value $value */
077⫶ $value = $content->getFieldValue(self::IMAGE_FIELD_IDENTIFIER);
078⫶
079⫶ if ($value === null || !$this->shouldGenerateAltText($value)) {
080⫶ $output->writeln(sprintf('Image %s has the image field empty or the alternative text is already specified. Skipping.', $content->getName()));
081⫶ continue;
082⫶ }
083⫶
084⫶ $contentUpdateStruct = $this->contentService->newContentUpdateStruct();
085⫸ $value->alternativeText = $this->getSuggestedAltText($this->convertImageToBase64($value->uri), $content->getDefaultLanguageCode());
086⫶ $contentUpdateStruct->setField(self::IMAGE_FIELD_IDENTIFIER, $value);
087⫶
088⫶ $updatedContent = $this->contentService->updateContent(
089⫶ $this->contentService->createContentDraft($content->getContentInfo())->getVersionInfo(),
090⫶ $contentUpdateStruct
091⫶ );
092⫶ $this->contentService->publishVersion($updatedContent->getVersionInfo());
093⫶ }
094⫶
095⫶ return Command::SUCCESS;
096⫶ }
097⫶
098⫸ private function getSuggestedAltText(string $imageEncodedInBase64, string $languageCode): string
099⫸ {
100⫸ $action = new GenerateAltTextAction(new Image([$imageEncodedInBase64]));
101⫸
102⫸ $action->setRuntimeContext(new RuntimeContext(['languageCode' => $languageCode]));
103⫸ $action->setActionContext(
104⫸ new ActionContext(
105⫸ new Options(['default_locale_fallback' => 'en']), // System context
106⫸ new Options(['max_lenght' => 100]), // Action Type options
107⫸ new Options( // Action Handler options
108⫸ [
109⫸ 'prompt' => 'Generate the alt text for this image in less than 100 characters.',
110⫸ 'temperature' => 0.7,
111⫸ 'max_tokens' => 4096,
112⫸ 'model' => 'gpt-4o-mini',
113⫸ ]
114⫸ )
115⫸ )
116⫸ );
117⫸
118⫸ $output = $this->actionService->execute($action)->getOutput();
119⫸
120⫸ assert($output instanceof Text);
121⫸
122⫸ return $output->getText();
123⫸ }
124⫶
125⫶ private function convertImageToBase64(?string $uri): string
126⫶ {
127⫶ $file = file_get_contents($this->projectDir . \DIRECTORY_SEPARATOR . 'public' . \DIRECTORY_SEPARATOR . $uri);
128⫶ if ($file === false) {
129⫶ throw new \RuntimeException('Cannot read file');
130⫶ }
131⫶
132⫶ return 'data:image/jpeg;base64,' . base64_encode($file);
133⫶ }
134⫶
135⫶ private function getModifiedImages(): ContentList
136⫶ {
137⫶ $filter = (new Filter())
138⫶ ->withCriterion(
139⫶ new DateMetadata(DateMetadata::MODIFIED, Operator::GTE, strtotime('-1 day'))
140⫶ )
141⫶ ->andWithCriterion(new ContentTypeIdentifier('image'));
142⫶
143⫶ return $this->contentService->find($filter);
144⫶ }
145⫶
146⫶ private function shouldGenerateAltText(Value $value): bool
147⫶ {
148⫶ return $this->fieldTypeService->getFieldType('ezimage')->isEmptyValue($value) === false &&
149⫶ $value->isAlternativeTextEmpty();
150⫶ }
151⫶
152⫶ private function setUser(string $userLogin): void
153⫶ {
154⫶ $this->permissionResolver->setCurrentUserReference($this->userService->loadUserByLogin($userLogin));
155⫶ }
156⫶}


code_samples/ai_actions/src/Form/Type/TextToTextOptionsType.php


code_samples/ai_actions/src/Form/Type/TextToTextOptionsType.php

docs/ai_actions/extend_ai_actions.md@128:``` php hl_lines="14-18"
docs/ai_actions/extend_ai_actions.md@129:[[= include_file('code_samples/ai_actions/src/Form/Type/TextToTextOptionsType.php') =]]
docs/ai_actions/extend_ai_actions.md@130:```

001⫶
002⫶
003⫶namespace App\Form\Type;
004⫶
005⫶use Symfony\Component\Form\AbstractType;
006⫶use Symfony\Component\Form\Extension\Core\Type\TextareaType;
007⫶use Symfony\Component\Form\FormBuilderInterface;
008⫶use Symfony\Component\OptionsResolver\OptionsResolver;
009⫶
010⫶final class TextToTextOptionsType extends AbstractType
011⫶{
012⫶ public function buildForm(FormBuilderInterface $builder, array $options): void
013⫶ {
014⫸ $builder->add('system_prompt', TextareaType::class, [
015⫸ 'required' => true,
016⫸ 'disabled' => $options['translation_mode'],
017⫸ 'label' => 'System message',
018⫸ ]);
019⫶ }
020⫶
021⫶ public function configureOptions(OptionsResolver $resolver): void
022⫶ {
023⫶ $resolver->setDefaults([
024⫶ 'translation_domain' => 'ibexa_connector_ai',
025⫶ 'translation_mode' => false,
026⫶ ]);
027⫶
028⫶ $resolver->setAllowedTypes('translation_mode', 'bool');
029⫶ }
030⫶}


code_samples/ai_actions/src/Form/Type/TranscribeAudioOptionsType.php


code_samples/ai_actions/src/Form/Type/TranscribeAudioOptionsType.php

docs/ai_actions/extend_ai_actions.md@188:``` php hl_lines="14-18"
docs/ai_actions/extend_ai_actions.md@189:[[= include_file('code_samples/ai_actions/src/Form/Type/TranscribeAudioOptionsType.php') =]]
docs/ai_actions/extend_ai_actions.md@190:```

001⫶
002⫶
003⫶namespace App\Form\Type;
004⫶
005⫶use Symfony\Component\Form\AbstractType;
006⫶use Symfony\Component\Form\Extension\Core\Type\CheckboxType;
007⫶use Symfony\Component\Form\FormBuilderInterface;
008⫶use Symfony\Component\OptionsResolver\OptionsResolver;
009⫶
010⫶final class TranscribeAudioOptionsType extends AbstractType
011⫶{
012⫶ public function buildForm(FormBuilderInterface $builder, array $options): void
013⫶ {
014⫸ $builder->add('include_timestamps', CheckboxType::class, [
015⫸ 'required' => false,
016⫸ 'disabled' => $options['translation_mode'],
017⫸ 'label' => 'Include timestamps',
018⫸ ]);
019⫶ }
020⫶
021⫶ public function configureOptions(OptionsResolver $resolver): void
022⫶ {
023⫶ $resolver->setDefaults([
024⫶ 'translation_domain' => 'ibexa_connector_ai',
025⫶ 'translation_mode' => false,
026⫶ ]);
027⫶
028⫶ $resolver->setAllowedTypes('translation_mode', 'bool');
029⫶ }
030⫶}


code_samples/ai_actions/src/Query/Search.php


code_samples/ai_actions/src/Query/Search.php

docs/search/ai_actions_search_reference/action_configuration_criteria.md@19:``` php
docs/search/ai_actions_search_reference/action_configuration_criteria.md@20:[[= include_file('code_samples/ai_actions/src/Query/Search.php') =]]
docs/search/ai_actions_search_reference/action_configuration_criteria.md@21:```

001⫶
002⫶
003⫶use Ibexa\Contracts\ConnectorAi\ActionConfiguration\ActionConfigurationQuery;
004⫶use Ibexa\Contracts\ConnectorAi\ActionConfiguration\Query\Criterion;
005⫶use Ibexa\Contracts\ConnectorAi\ActionConfiguration\Query\SortClause;
006⫶use Ibexa\Contracts\CoreSearch\Values\Query\AbstractSortClause;
007⫶use Ibexa\Contracts\CoreSearch\Values\Query\Criterion\FieldValueCriterion;
008⫶
009⫶$query = new ActionConfigurationQuery(
010⫶ new Criterion\LogicalAnd(
011⫶ new Criterion\Enabled(),
012⫶ new Criterion\LogicalOr(
013⫶ new Criterion\Name('Casual', FieldValueCriterion::COMPARISON_STARTS_WITH),
014⫶ new Criterion\Identifier('casual')
015⫶ )
016⫶ ),
017⫶ [
018⫶ new SortClause\Enabled(AbstractSortClause::SORT_DESC),
019⫶ new SortClause\Identifier(AbstractSortClause::SORT_ASC),
020⫶ ]
021⫶);
022⫶/** @var \Ibexa\Contracts\ConnectorAi\ActionConfigurationServiceInterface $actionConfigurationService */
023⫶$results = $actionConfigurationService->findActionConfigurations($query);

docs/search/ai_actions_search_reference/action_configuration_sort_clauses.md@14:``` php
docs/search/ai_actions_search_reference/action_configuration_sort_clauses.md@15:[[= include_file('code_samples/ai_actions/src/Query/Search.php') =]]
docs/search/ai_actions_search_reference/action_configuration_sort_clauses.md@16:```

001⫶
002⫶
003⫶use Ibexa\Contracts\ConnectorAi\ActionConfiguration\ActionConfigurationQuery;
004⫶use Ibexa\Contracts\ConnectorAi\ActionConfiguration\Query\Criterion;
005⫶use Ibexa\Contracts\ConnectorAi\ActionConfiguration\Query\SortClause;
006⫶use Ibexa\Contracts\CoreSearch\Values\Query\AbstractSortClause;
007⫶use Ibexa\Contracts\CoreSearch\Values\Query\Criterion\FieldValueCriterion;
008⫶
009⫶$query = new ActionConfigurationQuery(
010⫶ new Criterion\LogicalAnd(
011⫶ new Criterion\Enabled(),
012⫶ new Criterion\LogicalOr(
013⫶ new Criterion\Name('Casual', FieldValueCriterion::COMPARISON_STARTS_WITH),
014⫶ new Criterion\Identifier('casual')
015⫶ )
016⫶ ),
017⫶ [
018⫶ new SortClause\Enabled(AbstractSortClause::SORT_DESC),
019⫶ new SortClause\Identifier(AbstractSortClause::SORT_ASC),
020⫶ ]
021⫶);
022⫶/** @var \Ibexa\Contracts\ConnectorAi\ActionConfigurationServiceInterface $actionConfigurationService */
023⫶$results = $actionConfigurationService->findActionConfigurations($query);


code_samples/ai_actions/templates/themes/admin/admin/ui/fieldtype/edit/form_fields_binary_ai.html.twig


code_samples/ai_actions/templates/themes/admin/admin/ui/fieldtype/edit/form_fields_binary_ai.html.twig

docs/ai_actions/extend_ai_actions.md@289:``` twig
docs/ai_actions/extend_ai_actions.md@290:[[= include_file('code_samples/ai_actions/templates/themes/admin/admin/ui/fieldtype/edit/form_fields_binary_ai.html.twig') =]]
docs/ai_actions/extend_ai_actions.md@291:```

001⫶{% extends '@ibexadesign/ui/field_type/edit/ezbinaryfile.html.twig' %}
002⫶
003⫶{% block ezbinaryfile_preview %}
004⫶ {{ parent() }}
005⫶
006⫶ {% set transcriptFieldIdentifier = 'transcript' %}
007⫶ {% set fieldTypeIdentifiers = form.parent.parent.vars.value|keys %}
008⫶
009⫶ {% if transcriptFieldIdentifier in fieldTypeIdentifiers %}
010⫶ {% set module_id = 'TranscribeAudio' %}
011⫶ {% set ai_config_id = 'transcribe_audio' %}
012⫶ {% set container_selector = '.ibexa-edit-content' %}
013⫶ {% set input_selector = '.ibexa-field-edit-preview__action--preview' %}
014⫶ {% set output_selector = '#ezplatform_content_forms_content_edit_fieldsData_transcript_value' %}
015⫶ {% set cancel_wrapper_selector = '.ibexa-field-edit-preview__media-wrapper' %}
016⫶
017⫶ {% embed '@ibexadesign/connector_ai/ui/ai_module/ai_component.html.twig' with {
018⫶ ai_config_id,
019⫶ container_selector,
020⫶ input_selector,
021⫶ output_selector,
022⫶ } %}
023⫶ {% endembed %}
024⫶ {% endif %}
025⫶{% endblock %}


code_samples/ai_actions/webpack.config.js


code_samples/ai_actions/webpack.config.js

docs/ai_actions/extend_ai_actions.md@329:``` js
docs/ai_actions/extend_ai_actions.md@330:[[= include_file('code_samples/ai_actions/webpack.config.js', 40, 47) =]]
docs/ai_actions/extend_ai_actions.md@331:```

001⫶ ibexaConfig,
002⫶ entryName: 'ibexa-admin-ui-layout-js',
003⫶ newItems: [
004⫶ path.resolve(__dirname, './assets/js/addAudioModule.js')
005⫶ ],
006⫶});


code_samples/data_migration/examples/ai/action_configuration_create.yaml


code_samples/data_migration/examples/ai/action_configuration_create.yaml

docs/content_management/data_migration/importing_data.md@501:``` yaml
docs/content_management/data_migration/importing_data.md@502:[[= include_file('code_samples/data_migration/examples/ai/action_configuration_create.yaml') =]]
docs/content_management/data_migration/importing_data.md@503:```

001⫶- type: action_configuration
002⫶ mode: create
003⫶ identifier: foo_identifier
004⫶ enabled: false
005⫶ names:
006⫶ 'eng-GB': 'foo_name_eng_gb'
007⫶ 'ger-DE': 'foo_name_ger_de'
008⫶
009⫶ descriptions:
010⫶ 'ger-DE': 'foo_description_ger_de'
011⫶
012⫶ action_handler_identifier: foo_handler
013⫶ action_handler_options:
014⫶ handler_option: foo
015⫶
016⫶ action_type_identifier: generate_alt_text
017⫶ action_type_options:
018⫶ max_length: 130


code_samples/data_migration/examples/ai/action_configuration_delete.yaml


code_samples/data_migration/examples/ai/action_configuration_delete.yaml

docs/content_management/data_migration/importing_data.md@513:``` yaml
docs/content_management/data_migration/importing_data.md@514:[[= include_file('code_samples/data_migration/examples/ai/action_configuration_delete.yaml') =]]
docs/content_management/data_migration/importing_data.md@515:```

001⫶- type: action_configuration
002⫶ mode: delete
003⫶ match:
004⫶ field: identifier
005⫶ value: foo_identifier


code_samples/data_migration/examples/ai/action_configuration_update.yaml


code_samples/data_migration/examples/ai/action_configuration_update.yaml

@mnocon mnocon marked this pull request as ready for review November 12, 2024 08:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant